Systems and methods for interleaving series of medical images

ABSTRACT

After selecting two or more image series for comparison, images of the image series are interleaved so that they are alternatively displayed in a comparison pane on a display device. In one embodiment, after one or more image series are selected for comparison, an interleaved image series is created containing each of the images of the one or more selected image series, or, alternatively, the interleaved image series comprises links to the images arranged in the interleaved pattern. If differences exist in the images of the multiple image series, these differences may be more easily detectable as the display device cycles between the images. Comparison of images in an interleaved image series may be more advantageous if the images of each selected image series are of a common anatomical area, common image size, and the images are in the same order.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/502,055, filed on Sep. 30, 2014 and titled “SYSTEMS AND METHODS FORINTERLEAVING SERIES OF MEDICAL IMAGES,” which is a continuation of U.S.patent application Ser. No. 12/857,915, now U.S. Pat. No. 8,879,807,filed on Aug. 17, 2010 and titled “SYSTEMS AND METHODS FOR INTERLEAVINGSERIES OF MEDICAL IMAGES,” which is a continuation of U.S. patentapplication Ser. No. 11/268,261, now U.S. Pat. No. 7,885,440, filed onNov. 3, 2005 and titled “SYSTEMS AND METHODS FOR INTERLEAVING SERIES OFMEDICAL IMAGES,” which claims priority under 35 U.S.C. §119(e) to U.S.Provisional Application Ser. No. 60/625,690, filed on Nov. 4, 2004, eachof which is hereby expressly incorporated by reference in its entirety.

BACKGROUND OF THE INVENTION

Field of the Invention

This invention relates to management and viewing of medical images and,more particularly, to systems and methods of comparing related medicalimages in order to detect differences in the compared images.

Description of the Related Art

Medical imaging is increasingly moving into the digital realm. Thisincludes imaging techniques that were traditionally analog, such asmammography, x-ray imaging, angiography, endoscopy, and pathology, whereinformation can now be acquired directly using digital sensors, or bydigitizing information that was acquired in analog form. In addition,many imaging modalities are inherently digital, such as MRI, CT, nuclearmedicine, and ultrasound. Increasingly these digital images are viewed,manipulated, and interpreted using computers and related computerequipment. Accordingly, there is a need for improved systems and methodsof viewing and manipulating these digital images.

When comparison of related images is required, subtle differencesbetween images may be difficult to detect. For example, if a lungradiograph from two months previous, and a current lung radiograph areto be compared in order to determine if any changes have occurred in thelungs over the previous two months, the viewer or reader typically viewsthe two x-rays side by side. For example, the viewer or reader may havetwo monitors placed side by side, wherein each of the monitors displaysa chest radiographic image. Alternatively, the viewer may view the twoimages side by side on a single monitor. However, as those of skill inthe art will recognize, identifying differences in related images inthis manner is often tedious and difficult. Some imaging modalities,such as CT and MRI, produce a large number of images, hundreds to eventhousands of images per exam. In many cases, comparison of differentseries of images within the exam is required. For example, comparison ofpre and post contrast images to detect areas of enhancement orcomparison of PET and CT images for localization of activity is oftennecessary. Further, these often large exams may need to be compared tomultiple prior exams to detect subtle, progressive changes over time,for example to detect a small, growing tumor. Current imaging softwaredoes not provide a satisfactory method for comparing images contained intwo or more image series. Accordingly, systems and methods forcomparison of images of multiple image series so that differences in theimages may be more easily distinguishable are desired.

SUMMARY OF THE INVENTION

In one embodiment, a method of viewing medical images from two or moreimage series on a display device coupled to a computing device comprisesthe steps of selecting a first image series comprising two or moremedical images, selecting at least one comparison image series, each ofthe comparison image series comprising two or more medical images,interleaving images of the first image series and the comparison imageseries in order to form an interleaved image series, and sequentiallydisplaying the images of the interleaved image series at a singlelocation on the display device.

In another embodiment, a method of viewing a series of medical images ona display device coupled to a computing device comprises the steps of(a) selecting a first image series for viewing, the first image seriescomprising a plurality X of medical images, (b) selecting a second imageseries for viewing, the second image series comprising a plurality Y ofmedical images, (c) displaying at a predetermined location on thedisplay device a Nth image of the first image series, (d) replacing theNth image of the first image series with a Mth image of the second imageseries at the predetermined location, (e) incrementing N and M, and (f)repeating steps (c) to (f).

In another embodiment, a system for enhancing a viewer's ability todetect differences between medical images in two or more sets of medialimages comprises a display device, a graphical user interface displayedon the display device and comprising an image pane configured to displaya single medical image at a time, an image selection module to selecttwo or more sets of medical images, each of the sets of medical imagescomprising two or more medical images, and a user interface to receivecommands from a user, wherein in response to receiving a first commandfrom the user, the image pane sequentially displays a first medicalimage from each of the image sets and, after displaying the firstmedical image from each image set, the image pane sequentially displaysa second medical image from each image set. This process of displayingimages from images series alternatively continues through subsequentimages in the image series.

In another embodiment, a system of viewing medical images from two ormore image series on a display device coupled to a computing devicecomprises means for selecting a first image series comprising two ormore medical images, means for selecting at least one comparison imageseries, each of the comparison image series comprising two or moremedical images, means for interleaving images of the first image seriesand the comparison image series in order to form an interleaved imageseries, and means for sequentially displaying the images of theinterleaved image series at a single location on the display device.

In another embodiment, a method of forming an interleaved image seriescomprises selecting N groups of images, each of the groups of imagescomprising two or more images, determining a starting image of each ofthe groups of images, creating an interleaved image series comprisingimages from each of the selected N groups of images, wherein the imagesof the interleaved image series are ordered so that an image from eachof the N groups of images is included in each sequential Nth group ofimages, and providing the interleaved image series to a user interfacefor sequential display in a predetermined location of a display device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an exemplary computing system incommunication with a network and various networked devices.

FIG. 2 is a diagram illustrating a display device having images from twoimage series concurrently displayed in image panes displayed on adisplay device.

FIG. 3 is a diagram illustrating a display device having images fromfour image series concurrently displayed in image panes.

FIG. 4 is a diagram illustrating sequential changes to a comparison panedisplayed on the display device as images from two image series arecompared.

FIG. 5 is a diagram illustrating an exemplary interleaving of four imageseries.

FIG. 6 is a flowchart illustrating an exemplary method of viewing imagesfrom multiple image series.

FIGS. 7A and 7B are images of a first image series.

FIGS. 8A and 8B are images of a second image series.

FIGS. 9A-9D are the images of the first and second image seriesinterleaved for sequential viewing in a predetermined portion of adisplay.

FIG. 10 is a diagram illustrating a display device displaying twocomparison panes.

FIG. 11 illustrates the first images of the first and second imageseries illustrated in FIGS. 7A and 8A interleaved for alternativeviewing in a comparison pane.

FIG. 12 illustrates the second images of the first and second imageseries illustrated in FIGS. 7B and 8B interleaved for alternativeviewing in a comparison pane.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

Embodiments of the invention will now be described with reference to theaccompanying figures, wherein like numerals refer to like elementsthroughout. The terminology used in the description presented herein isnot intended to be interpreted in any limited or restrictive manner,simply because it is being utilized in conjunction with a detaileddescription of certain specific embodiments of the invention.Furthermore, embodiments of the invention may include several novelfeatures, no single one of which is solely responsible for its desirableattributes or which is essential to practicing the inventions hereindescribed.

FIG. 1 is a block diagram of an exemplary computing system 100 incommunication with a network 160 and various network devices. Thecomputing system 100 may be used to implement certain systems andmethods described herein. The functionality provided for in thecomponents and modules of computing system 100 may be combined intofewer components and modules or further separated into additionalcomponents and modules.

The computing system 100 includes, for example, a personal computer thatis IBM, Macintosh, or Linux/Unix compatible. In one embodiment, theexemplary computing system 100 includes a central processing unit(“CPU”) 105, which may include a conventional microprocessor, anapplication module 145 that comprises one or more various applicationsthat may be executed by the CPU 105. The application module 145 mayinclude, by way of example, components, such as software components,object-oriented software components, class components and taskcomponents, processes, functions, attributes, procedures, subroutines,segments of program code, drivers, firmware, microcode, circuitry, data,databases, data structures, tables, arrays, and variables.

The computing system 100 further includes a memory 130, such as randomaccess memory (“RAM”) for temporary storage of information and a readonly memory (“ROM”) for permanent storage of information, and a massstorage device 120, such as a hard drive, diskette, or optical mediastorage device. Typically, the modules of the computing system 100 areconnected to the computer using a standards-based bus system. Indifferent embodiments of the present invention, the standards based bussystem could be Peripheral Component Interconnect (PCI), Microchannel,SCSI, Industrial Standard Architecture (ISA) and Extended ISA (EISA)architectures, for example.

The computing system 100 is generally controlled and coordinated byoperating system software, such as the Windows 95, 98, NT, 2000, XP orother compatible operating systems. In Macintosh systems, the operatingsystem may be any available operating system, such as MAC OS X. In otherembodiments, the computing system 100 may be controlled by a proprietaryoperating system. Conventional operating systems control and schedulecomputer processes for execution, perform memory management, providefile system, networking, and I/O services, and provide a user interface,such as a graphical user interface (“GUI”), among other things.

The exemplary computing system 100 includes one or more of commonlyavailable input/output (I/O) devices and interfaces 110, such as akeyboard, mouse, touchpad, and printer. In one embodiment, the I/Odevices and interfaces 110 include one or more display devices, such asa monitor, that allows the visual presentation of data to a user. Moreparticularly, display devices provide for the presentation of GUIs,application software data, and multimedia presentations, for example. Inone embodiment, a GUI includes one or more image panes in which medicalimages may be displayed. As described in further detail below, a GUI mayprovide a comparison pane on a display device in which images frommultiple image series are sequentially displayed. According to thesystems and methods described below, medical images may be stored on thecomputing system 100 or another device that is local or remote,displayed on a display device, and manipulated by the application module145. The computing system 100 may also include one or more multimediadevices 140, such as speakers, video cards, graphics accelerators, andmicrophones, for example.

In the embodiment of FIG. 1, the I/O devices and interfaces 110 providea communication interface to various external devices. In the embodimentof FIG. 1, the computing system 100 is coupled to a network 160, such asa LAN, WAN, or the Internet, for example, via a communication link 115.The network 160 may be coupled to various computing devices and/or otherelectronic devices. In the exemplary embodiment of FIG. 1, the network160 is coupled to imaging devices 170, an image server 180, and amedical facility 190. In addition to the devices that are illustrated inFIG. 1, the network 160 may communicate with other computing, imaging,and storage devices.

The imaging devices 170 may be any type of device that is capable ofacquiring medical images, such as a MRI, x-ray, mammography, or CT scansystems. The image server 180 includes a data store 182 that isconfigured to store images and data associated with images. In oneembodiment, the imaging devices 170 communicate with the image servervia the network 160 and image information is transmitted to the imageserver 160 and stored in the data store 182. In one embodiment, theimage data is stored in Digital Imaging and Communications in Medicine(“DICOM”) format. The complete DICOM specifications may be found on theNational Electrical Manufactures Association Website at<medical.nema.org>. Also, NEMA PS 3—Digital Imaging and Communicationsin Medicine, 2004 ed., Global Engineering Documents, Englewood Colo.,2004, provides an overview of the DICOM standard. Each of theabove-cited references is hereby incorporated by reference in theirentireties. In one embodiment, the data store 182 also stores theuser-defined display parameters associated with one or more of theimages stored on the data store 182. As discussed in further detailbelow, the user-defined display parameters may vary depending of thetype of image, area imaged, clinical indication, source of image,display device, user, or other factors. Accordingly, any type ofuser-defined display parameter is expressly contemplated for use inconjunction with the systems and methods described herein.

The exemplary image server 180 is configured to store images frommultiple sources and in multiple formats. For example, the image server180 may be configured to receive medical images in the DICOM format frommultiple sources, store these images in the data store 182, andselectively transmit medical images to requesting computing devices.

The medical facility 190 may be a hospital, clinic, doctor's office, orany other medical facility. The medical facility 190 may include one ormore imaging devices and may share medical images with the image server180 or other authorized computing devices. In one embodiment, multiplecomputing systems, such as the computing system 100 may be housed at amedical facility, such as medical facility 190.

Definition of Terms

“Medical image” is defined to include an image of an organism. It mayinclude but is not limited to a radiograph, computed tomography (CT),magnetic resonance imaging (MRI), Ultrasound (US), mammogram, positronemission tomography scan (PET), nuclear scan (NM), pathology, endoscopy,ophthalmology, or many other types of medical images. While thisdescription is directed to viewing and tracking of medical images, themethods and systems described herein may also be used in conjunctionwith non-medical images, such as, images of circuit boards, airplanewings, and satellite images, for example.

“Modality” is defined as a medical imaging device (a patient whoundergoes an MRI is said to have been scanned with the MRI modality).

“Image series” refers to two or more images that are related. Forexample, an image series may comprise two or more images of a particularpatient that are acquired on a particular date, e.g., different x-rayprojections of the chest. A series of contiguous 3 mm axial CT scans ofthe chest would be another example of an image series. A brain MRI scanmight include the following series: sagittal T1 weighted images, axialT1 weighted images, axial FLAIR images, axial T2 weighted images, aswell as post contrast axial, sagittal and coronal T1 weighted series. Animage series may be limited to images of a certain modality or maycomprise images of multiple modalities.

“Patient” refers to an individual who undergoes a medical imagingexamination.

“Display parameters” are defined to include methods of display of animage or exam. Display parameters may include, for example, a pixelwindow level and width (similar to brightness and contrast), a certaincolor map that renders different pixel intensities as different colors,or opacity map.

“Interleaving,” is defined to include the process of arranging imagesfrom multiple image series by regularly alternating between images ofthe multiple image series in order to create a resultant “interleaved”image series. In one embodiment, an interleaved image series comprisesimages from multiple image series ordered so that the interleaved imageseries alternates between the images of the original series. Forexample, when image series A comprising images A1, A2, . . . An, imageseries B comprising images B1, B2, . . . Bn, and image series Ccomprising images C1, C2, . . . Cn are interleaved, the resultantinterleaved image series is ordered: A1, B1, C1, A2, B2, C2, . . . An,Bn, Cn. Images from multiple image series may be interleaved in variouspatterns and multiple interleaved image series may be generated from twoor more image series.

FIG. 2 is a diagram illustrating a display device having images from twoimage series concurrently displayed in image panes 212, 214 displayed ona display device 210. In the discussion that follows, the display device210 is coupled to a computing device, such as computing device 100, andreceives display information from the computing device 100. While thesystems and methods described below for interleaving and viewing imagesof multiple image series may be controlled by any suitable computingdevice, for ease of explanation herein, reference will be made to adisplay device coupled to computing device 100.

In the embodiment of FIG. 2, an image of a series A image series isdisplayed in the image pane 212, while an image of the series B imageseries is displayed in the image pane 214. As noted above, each imageseries comprises a group of images that are related in some way, such ashaving been acquired from a patient on a particular day. Although only asingle image of each of the image series is simultaneously displayed onthe display device 210, the series A and series B image series eachcomprise multiple images.

In certain embodiments, such as where the series A and B images arerelated, e.g., series A comprises mammogram images of a patient taken ona first date and series B comprises the mammogram images of the samepatient taking on a later date, it may be advantageous to identifydifferences between the images of Series A and Series B. However, asdescribed above, it is difficult to distinguish minor or smalldifferences between images using currently available image comparisontechniques. Using current image comparison systems, if a lung radiographfrom two months previous, and a current lung radiograph are to becompared in order to determine if any changes have occurred in the lungsover the previous two months, the viewer or reader typically views thetwo x-rays side by side, such as in image panes 212, 214 illustrated inFIG. 2. As those of skill in the art will recognize, identifyingdifferences in related images in this manner is often tedious anddifficult. Accordingly, described hereinafter are exemplary systems andmethods for comparison of images of multiple image series so thatdifferences between (in contrast to within) the images may be moreeasily distinguishable. In certain embodiments, related images aredisplayed sequentially in a single viewing pane on a display device sothat difference between the images may be more easily detected. Thesystems and methods described herein are applicable to any two or moreimages, including multiple images of multiple image series, for example.

FIG. 3 is a diagram illustrating a display device having images fromfour image series concurrently displayed in image panes 312, 314, 316,in 318. In the embodiment of FIG. 3, the image pane 312 displays imagesfrom series A, the image pane 314 displays images from series B, theimage pane 316 displays images from series C, and the image pane 318displays images from series D. Thus, a single image from each of thefour image series A-D is concurrently displayed on the display device210. In certain prior art systems, comparison of images of multipleseries was performed using a graphical user interface such as displayedin FIG. 3, wherein the user distinguishes differences between imagesthat are displayed side-by-side on a display device.

FIG. 4 is a diagram illustrating sequential changes to a comparison pane410 displayed on the display device 210 as images from two image seriesare compared. The comparison pane 410 is configured to display a singleimage. In one embodiment, one of the display panes, e.g. display pane312 on display device 210, serve as the comparison pane 410. In theembodiment of FIG. 4, images from two image series, images series A andB, have been selected for comparison. After being selected forcomparison using any suitable selection method, images from series A andB are interleaved so that they are alternatively displayed in thecomparison pane 410.

In embodiments with more than two series, the interleaved images may beordered according to various schemes. For example images from four imageseries may be ordered as follows: first image of first image series,first image of second image series, first image of third image series,first image of fourth image series, second image of first image series,and so forth. In other embodiments, however, the interleaved images maybe ordered differently. For example, images from four image series mayalso be ordered as follows: first image of first image series; firstimage of second image series; first image of first image series; firstimage of third image series; first image of first image series; andfirst image of fourth image series. Any other ordering of images frommultiple image series falls within the scope of “interleaving” as usedherein.

FIG. 4 shows the display device 210 at four steps 402, 404, 406, 408 ofthe comparison process, where the comparison process describes theprocess of displaying images in an interleaved image series. Moreparticularly, in step 402, a first image of image series A is displayedin the comparison pane 410. Moving to step 404, the first image of imageseries A is replaced by a first image of image series B in thecomparison pane 410. Assuming the images of series A and B are of thesame subject, the image displayed in steps 402 and 404 may be verysimilar. Accordingly, if differences exist in the first images of seriesA and B, these differences may be more easily detectable as the displaydevice cycles between the two images. Comparison of images in aninterleaved image series may be more advantageous if the images of eachselected image series are of a common anatomical area, common imagesize, common image orientation, and the images are in the same order.

In one embodiment, the computing system 100 that is coupled to thedisplay 210 may store settings for displaying images of particular imageseries, such as, for example, time for displaying each image, resolutionof each image, cropping to be applied to each image, and any othersetting that maybe appropriate. In one embodiment, the time fordisplaying an image may be determined real time by the user. Forexample, the user may press a designated key on a keyboard or mouse inorder to indicate that the current image should be replaced with anadjacent image in the interleaved image series. In another embodiment,the user selects settings for display of the images. For example, theuser may select an appropriate zoom level of an image series that shouldbe applied to each image in the image series.

Thus, the images of series A may be magnified more or less than theimages of series B. In addition, the user may adjust any othervisualization settings for individual images, an entire image series, ortwo or more image series.

With the first image of series B displayed in the comparison pane 410(step 404), the user may initiate viewing of an adjacent image in theinterleaved image series by pressing a certain key on a keyboard ormouse, for example. In one embodiment, a first input from a mouseindicates that a next image, e.g. image 2 of series A (step 406) shouldbe displayed in the comparison pane 410 and a second input from themouse indicates that a previous image, e.g. image 1 of series A (step402) should again be displayed in the comparison pane 410. In oneembodiment, the first input is entered by the user moving a scrollbutton on the mouse in a first direction and the second input is enteredby the user moving the scroll button on the mouse in an oppositedirection. Thus, the user may change the content of the comparison pane410 to either a next or a previous image in the interleaved imageseries. For example, at step 404, if the user wishes to again view thefirst image of series A, e.g., in order to locate difference in thefirst images of series A and B, the user may provide an input to thecomputing device 100 indicating movement to a previous image.Alternatively, at step 404, if the user wishes to view a next image inthe interleaved image series, the user may provide an input to thecomputing device 100 indicating movement to a next image.

At step 406, the second image of series A is displayed in the comparisonpane 410, replacing the first image of series B (step 404). At step 406,the user may provide inputs to the computing device 100 indicating thatthe comparison pane 410 should be updated with a previous image, e.g.step 404, or a subsequent image, e.g., step 408.

At step 408, the second image of series B is displayed in the comparisonpane 410, replacing the second image of series B (step 406). At step406, the user may provide inputs to the computing device 100 indicatingthat the comparison pane 410 should be updated with a previous image,e.g. step 404, or a subsequent image. In one embodiment, each of theimage series A and B include more than two images, such as 3 or moreimages, and the images of series A and B are displayed in the mannerdescribed above with respect to FIG. 4. In one embodiment, more than twoimages series may be interleaved for display in the comparison pane. Forexample, if three images series, e.g., series A, B, and C, are selectedfor comparison, a first image of each of the series will be sequentiallydisplayed in the comparison pane, followed by a second image of each ofthe series, and so on. As noted above, the user may control the timingof transition between display of images in the interleaved image seriesand may even control the direction of movement in the interleavedseries. Additionally, the user may control alignment and/or positioningof the images of each images series in order to precisely aligninterleaved images from multiple series.

In one embodiment, the images of each of the image series areautomatically modified so that characteristics of the images aresimilar. For example, images may be adjusted by changing their size,rotation, and location. If the images are of substantially the sameanatomical structure, when the images of the interleaved image seriesare displayed in the comparison pane, differences between adjacentimages may be more easily detected. In one embodiment, selected imagesare morphed in order to achieve a common size of the anatomicalstructure of interest in each of the images. In one embodiment,photographic filters may be applied to all images of one or more imageseries, or to selected images of one or more image series, to furtherenhance the viewer's ability to distinguish differences in the images.

In one embodiment, information regarding the image currently displayedin the comparison pane 410 is displayed on the display device 210 andupdated as the images in the comparison pane 410 are changed. Forexample, information regarding the images series and image number withinthe series may be displayed for each image. In addition, the exam dateand time may also be displayed and updated as the images of theinterleaved image series are displayed in the comparison pane 410. Inone embodiment, an indicator of whether the current display is of aninterleaved image series or a single image series is displayed on thedisplay device. For example, “interleaved” may be displayed at the topof the display device when an interleaved image series is displayed in acomparison pane. In some embodiment, the user chooses what informationrelated to the images of the interleaved image series should bedisplayed. The user may also be provided the ability to turn the displayof information on and off, such as by pressing a particular key or keycombination on the keyboard.

FIG. 5 is a diagram illustrating an exemplary interleaving of four imageseries in creating an interleaved image series. As noted above, theimage series that are selected for comparison, and thus, are selectedfor interleaving, may be selected by a user in one of several manners ormay be automatically selected by the computing device based onproperties of the image series. In one embodiment, the multiple imageseries are interleaved so that a first image of each series is displayedin the comparison pane prior to display of a second image of any of theother selected image series. FIG. 5 illustrates an exemplaryinterleaving of four image series, series A, B, C, and D, eachcomprising four images. As those of skill in the art will recognize,more or less image series, each comprising more or less images, may beinterleaved in a manner similar to that illustrated in FIG. 5.

In FIG. 5, the images of series A are represented by diamonds in acolumn 510, where a number in each of the diamonds represents a specificimage within image series A. Similarly, images of series B arerepresented by squares in column 520, series C are represented by ovalsin a column 530, and series D are represented by parallelograms in acolumn 540. Each of the images in images series B, C, and D are likewisenumbered 1-4, indicating a particular image in each image series. Asnoted above, the first image selected for comparison in a particularimage series may not be the first image in the image series, e.g., thefirst image of an exam. Thus, although each image series A-D begins witha first image labeled image “1”, this first image may not be the firstimage in the image series, but may be a user selected, or automaticallyselected, start image. In addition, each of the images may haveuser-defined display parameters that are different than other images ofthe same series and/or other image series. In some embodiments, displayparameters, such as zoom level, cropping, and color characteristics, maybe simultaneously changed for each image in an image series, such asseries A, B, C, D, or an interleaved image series, such as interleavedimage series 540.

As illustrated in FIG. 5, an interleaved image series 550 each of theimages 1-4 in each of the image series A-D. More particularly, theinterleaved image series 550 comprises a first image from each of seriesA-D, followed by a second image from each of the series A-D, followed bya third image from each of the series A-D, followed by a fourth imagefrom each of the series A-D. Thus, when the interleaved image series 550is displayed in the comparison pane, a first image of the image series Ais displayed, followed by a first image of image series B, a first imageof image series C, and a first image of image series D. While the orderof the interleaved image series 550 is maintained during viewing of theinterleaved images, the direction of movement between adjacent imagesmay be selected by the user or automatically by the computing device100.

FIG. 6 is a flowchart illustrating an exemplary method of viewing imagesfrom multiple image series. Using the method of FIG. 6, multiple imageseries may be easily compared and differences between images of themultiple image series may be distinguished.

In a block 610, a first image series is selected. As noted above, animage series is a group of two or more images that are in some wayrelated. For example, a first image series may comprise multiple chestx-rays of a patient that are taken on a given date.

In a block 620, one or more comparison image series are selected. Theseimage series also each comprise two or more images that are in some wayrelated, such as having been taken at a common exam. The comparisonimage series should be related to the first image series so that whenthe first image series and the comparison image series are compared,meaningful distinctions between the image series may be detected.

In one embodiment, the first image series and the comparison imageseries are selected by the user clicking on a button indicating thatimage interleaving is desired. In one embodiment, a user right-clickswith a mouse on an image of a first image series in order to initiatedisplay of an “interleave menu” listing options for selecting imageseries for interleaving and viewing the interleaved image series. In oneembodiment, the interleave menu includes an option, such as “interleaveadjacent,” indicating that the user may select one of the other imageseries displayed on the display for interleaving. In certainembodiments, any number of image panes may be simultaneously displayedon the display device 210. For example, in FIG. 2, two image panes212,214 are display and in FIG. 3, four image panes 312, 314, 316, 318are displayed. In other embodiment, six, eight, ten, twelve, or moreimage panes may be concurrently displayed on the display device.

When the interleave adjacent option is selected, the user may select oneor more comparison series by moving the pointer to a border between theadjacent series and clicking the mouse button. In one embodiment, thecursor icon changes when it is positioned in a border indicating thatthe adjacent image series may be selected for comparison by clicking themouse button. With reference to FIG. 3, for example, the user may rightclick on the image pane 314 in order to select series B as the firstimage series and to initiate display of the interleave menu. From theinterleave menu, if the user selects interleave adjacent, the user maythen move the pointer to the border between the image panes 312 and 314and click the mouse button in order to select series A as a comparisonimage series. In one embodiment, selecting a comparison image seriesinitiates creation of an interleaved image series and displays the firstimage of the interleaved image series in the comparison pane. In anembodiment when only two image series are represented on the displaydevice, such as FIG. 2, selection of interleave adjacent from theinterleave menu may automatically select the two displayed image seriesfor interleaving and initiate creation and viewing of a interleavedimage series.

In one embodiment, the interleave menu also includes an option thatallows the user to select an adjacent image series for interleaving and,after selection of the first and comparison image series, displaysimages of the interleaved images series in a comparison pane that coversthe entire display area, or substantially all of the display area, ofthe display device. In this way, the images of the selected image seriesmay be viewed at a higher magnification level and, accordingly,differences in the images may be more easily detectable. Thus, in anembodiment that displays four image panes on the display device (e.g.,FIG. 3), after selection of this option from the interleave menu, asingle click on a comparison image series may cause the computing deviceto generate an interleaved image series and display a first image of theinterleaved image series in a comparison pane that covers substantiallyall of the display area of the display device, e.g., the area previouslycovered by the four image panes 312, 314, 316, 318 or FIG. 3.Advantageously, this “interleave and jump to full screen display” optionon the interleave menu provides an efficient transition from display ofmany image series to the display of a single interleaved series in acomparison pane that covers all, or substantially all, of the displayarea of a display device.

In one embodiment, the interleave menu includes an option that initiatesautomatic selection of one or more comparison image series based uponcharacteristics of the selected first image series. For example, imageseries with the same or similar names may be selected as comparisonimage series. In addition, image series may be selected automaticallybased upon any other criteria, such as one or more information itemscontained in the DICOM headers of images. In one embodiment, when thisoption is chosen from the interleave menu, a list of image series thathave the same series name, or other criteria that may be user defined,may be displayed. The user may then select one or more of the displayedseries as comparison image series.

The interleave menu advantageously allows the user to select imageseries for interleaving and automatically display the generatedinterleaved image series with minimal input from the user. For example,after selecting “interleave adjacent” on the interleave menu, a singleclick of a mouse, for example, on a border between the images to beinterleaved causes the computing system 100 to generate an interleavedimage series and display a first image of the interleaved image seriesin a comparison pane on the display device.

Returning to the diagram of FIG. 6, in a block 630, a starting image ofeach of the image series is selected. In one embodiment, a first imageof the first image series and each of the comparison image series aredisplayed on a display device. A user, using an input device, such as amouse or keyboard, may cycle through the images in each of the imageseries in order to determine a first image for comparison. For example,images of certain modalities, such as CT and MRI images, may not havestarting images that are each taken at similar physical locations withinthe patient. Thus, in these embodiments the user may select a differentstarting image in each of the image series so that adjacent images inthe interleaved image series are more closely related. For example, ifthe images series to be interleaved are series A comprising images A1,A2, . . . An, series B comprising images B1, B2, . . . Bn, and series Ccomprising images C1, C2, . . . Cn, the user may select images A1, B3,and C5 as the starting images of the respective image series so that theresultant interleaved image series is ordered A1, B3, C5, A2, B4, C6,A3, B5, C7, . . . Ax, Bx+2, Cx+4.

In an advantageous embodiment, the starting image in each of the seriesshould be related so that meaningful differences between the images ofthe image series are detectable. In one embodiment, the user may adjustdisplay characteristics of each image in an image series by adjustingthe display characteristics of the image currently displayed from thedesired image series. For example, if the first image series is at ahigher zoom level than the comparison image series, the zoom level ofeach image in the first image series may be adjusted by adjusting thezoom level of the currently display image of the first image series.

After selecting the first image series and the comparison image series,and selecting the display characteristics of one or more of the imageswhich are then applied to the other images in the series to which itbelongs, the image series are interleaved so that an interleaved imageseries is created, as illustrated in FIG. 5, for example.

In a block 640, images from the interleaved image series are displayedin a comparison pane displayed on the display device. In one embodiment,the comparison pane fills substantially the entire display area of adisplay device. In another embodiment, the comparison pane is smallerthan a total display area of the display device. In one embodiment, auser input determines when a current image displayed in the comparisonpane is updated with an adjacent image, by moving a scroll wheel on amouse while pressing a mouse button, for example. In one embodiment,when the user has completed viewing the interleaved image series, theprevious layout on the display device may be restored by the userperforming a specific action, such as releasing the mouse button that isdepressed while viewing interleaved images.

FIGS. 7A and 7B are images of a first image series, FIGS. 8A and 8B areimages of a second image series, and FIGS. 9A-9D are the images of aninterleaved image series comprising images of the first and second imageseries. More particularly, FIGS. 7A and 7B are two mammographic images710, 720 in a first image series. In one embodiment, the images 710, 720are of a patient on a first date. FIGS. 8A and 8B are two mammographicimages 810, 820 in a second image series. In one embodiment, the images710, 720, 810, 820 are of the same patient, but the images 810, 820 weretaken at a later date than the images 710, 720. Thus, differencesbetween the earlier mammographic images 710, 720 and the latermammographic images 810, 820 may be useful in detection and diagnosis ofthe patient. Accordingly, comparison of the images of the images seriesillustrated in FIGS. 7 and 8 may allow detection of differences betweenthe earlier and later images.

FIGS. 9A-9D are the images of FIGS. 7 and 8 combined in an interleavedimage series for viewing in a comparison pane. For example, image 710may first be displayed in a comparison pane. When the user indicatesthat a next image should be displayed, image 810 may replace image 710in the comparison pane. With image 810 displayed in the comparison pane,when the user indicates that a next image should be displayed, image 720is replaced with image 810 in the comparison pane. With image 720displayed in the comparison pane, when the user indicates that a nextimage should be displayed, image 720 is replaced with image 820 in thecomparison pane. As noted above, however, movement between images in aninterleaved image series, such as that of FIG. 9, may be bidirectionalso that the user may move back and forth between images as desired.

In another embodiment, interleaving of image series produces two or moreinterleaved image series. In one embodiment, the first image of eachimage series may be interleaved and alternatively displayed in a firstcomparison pane of the display device, while the second image of eachimage series may be interleaved and alternatively displayed in a secondcomparison pane of the display device, and so on. For example, if theimage series to be interleaved are series A comprising images A1, A2,and series B comprising images B1, B2, the system may generate a firstinterleaved series ordered A1, B1, and a second interleaved image seriesordered A2, B2. In one embodiment, images from each from each of thefirst and second interleaved image series are concurrently displayed ona display device in separate comparison panes. In one embodiment, anoption on the interleave menu, discussed above, may be selected in orderto initiate generation of multiple interleaved image series andconcurrent display of multiple comparison panes on the display device.In one embodiment, more than two images of image series may beinterleaved in the above-described manner and more than two comparisonpanes may be displayed concurrently on a display device for viewing thegenerated interleaved image series.

FIG. 10 is a diagram illustrating the display device 210 having twocomparison panes 1010A, 1010B. As noted above, multiple interleavedimage series may be concurrently displayed on the display device 210 inseparate comparison panes. In another embodiment, the display device 210displays three or more comparison panes for concurrently displayingthree or more interleaved image series.

FIG. 11 illustrates the first images 710, 810 of the first and secondimage series illustrated in FIGS. 7A and 8A and FIG. 12 illustrates thesecond images 720, 820 of the first and second image series illustratedin FIGS. 7B and 8B. The images 710, 810 (FIG. 11) comprise a firstinterleaved image series, while the images 720, 820 (FIG. 12) comprise asecond interleaved image series. In the embodiment illustrated in FIGS.11 and 12, the images 710, 720 are from a first exam and the images 810,820 are from a second exam. However, the first selected images from eachexam, e.g., images 710, 810, are of a first projection, while the secondselected images from each exam, e.g., 720, 820, are of a secondprojection. Thus, it may be advantageous for an interpreter of theimages to view the images of the same projection, from different exams,in separate interleaved image series. Accordingly, in the embodiment ofFIGS. 10, 11, and 12, a first interleaved image series comprising images710, 810 are viewed in a first comparison pane 1010A while a secondinterleaved image series comprising images 720, 820 are viewed in asecond comparison pane 1010B. In this embodiment, the viewer mayadvantageously move between images of the same projection in acomparison pane in order to identify differences in the images, whilealso viewing images of one or more additional projections in additionalcomparison panes. In one embodiment, any number of images may beincluded in each of the interleaved images series displayed incomparison panes 101A, 1010B, and additional comparison panes may beconcurrently displayed on the display device 210.

The foregoing description details certain embodiments of the invention.It will be appreciated, however, that no matter how detailed theforegoing appears in text, the invention can be practiced in many ways.As is also stated above, it should be noted that the use of particularterminology when describing certain features or aspects of the inventionshould not be taken to imply that the terminology is being re-definedherein to be restricted to including any specific characteristics of thefeatures or aspects of the invention with which that terminology isassociated. The scope of the invention should therefore be construed inaccordance with the appended claims and any equivalents thereof.

What is claimed is:
 1. A method comprising: receiving, at a computingsystem, at least a first plurality of images comprising a first imageseries and a second plurality of images comprising a second imageseries, wherein the first plurality of images and the second pluralityof images together comprise a set of images; analyzing, by the computingsystem, the set of images to identify values of a first characteristicassociated with respective images of the set of images, wherein thefirst characteristic comprises a time at which the respective images ofthe set of images were obtained; determining, by the computing system,at least a second characteristic associated with each of the images ofthe set of images; determining, by the computing system, an ordering ofthe images of the set of images based on: first: values of the firstcharacteristic associated with respective images of the set of images,and second: values of the second characteristic associated withrespective images of the set of images; displaying at a location of adisplay, a first image of the set of images, that is first in thedetermined ordering of the images; and in response to input from a userof the computing system indicating a direction of movement within theset of images, replacing, at the location, the first image with a secondimage, wherein the second image is adjacent to the first image in theordering of the images.
 2. The method of claim 1, wherein the secondcharacteristic comprises at least one of: a modality, an application ofcontrast agent, or an anatomical position associated with the respectiveimages.
 3. The method of claim 2, wherein the ordering of the images isfurther determined based on at least one of a type of image, an areaimaged, a clinical indication, a source of image, a display device, or auser.
 4. The method of claim 2, wherein the ordering of the images isfurther determined based on at least one of brightness, contrast, size,opacity map, rotation, location, zoom level, cropping, morphing, orcolor of the images.
 5. The method of claim 1, wherein said analyzingcomprises determining the times by reference to headers of therespective images.
 6. The method of claim 1, wherein a displaycharacteristic of the second image is adjusted such that the informationdisplayed in the first image and the second image more closely matches,and differences between the first image and the second image are moreeasily detectable by the user.
 7. The method of claim 6, wherein thedisplay characteristic comprises at least one of brightness, contrast,size, opacity map, rotation, location, zoom level, cropping, morphing,or color.
 8. The method of claim 1, wherein the computing system isconfigured to provide the input in response to the user moving a scrollwheel of a mouse or other input device in a first direction.
 9. Themethod of claim 1 further comprising: in response to a second input fromthe user of the computing system indicating an opposite direction ofmovement within the set of images, replacing, at the location, thesecond image with the first image.
 10. The method of claim 9, whereinthe computing system is configured to provide the second input inresponse to the user moving a scroll wheel of a mouse or other inputdevice in a first direction.
 11. The method of claim 10, wherein thecomputing system is configured to provide the second input in responseto the user moving a scroll wheel of a mouse or other input device in asecond direction.
 12. The method of claim 1 further comprising:determining, by the computing system, a characteristic associated withthe first image series; identifying, by the computing system, the secondimage series for comparison with the first image series, wherein thesecond image series is identified based on the second image series beingassociated with the same characteristic as the first image series. 13.The method of claim 12, wherein the characteristic associated with thefirst image series comprises at least one of: a header information item,a filename, a modality, a name, an anatomical area, an image size, or animage orientation.
 14. The method of claim 1, further comprising inresponse to another input from the user of the computing systemindicating the direction of movement within the set of images,replacing, at the location, a third image in the images of the set ofimages with another image of the set of images that is adjacent to thethird image in the ordering of the images.
 15. A computing systemcomprising: an electronic display; an input device; a non-transitorycomputer-readable storage medium configured to store softwareinstructions; and one or more computer processors in communication withthe electronic display, the input device, and the non-transitorycomputer-readable medium, the one or more computer processors configuredto execute the software instructions in order to cause the computingsystem to: receive at least a first plurality of images comprising afirst image series and a second plurality of images comprising a secondimage series, wherein the first plurality of images and the secondplurality of images together comprise a set of images; analyze the setof images to identify values of a first characteristic associated withrespective images of the set of images, wherein the first characteristiccomprises a time at which the respective images of the set of imageswere obtained; determine at least a second characteristic associatedwith each of the images of the set of images; determine an ordering ofthe images of the set of images based on: first: values of the firstcharacteristic associated with respective images of the set of images,and second: values of the second characteristic associated withrespective images of the set of images; display a location of a display,a first image of the set of images, that is first in the determinedordering of the images; and in response to input from a user of thecomputing system indicating a direction of movement within the set ofimages, replace, at the location, the first image with a second image,wherein the second image is adjacent to the first image in the orderingof the images.
 16. A non-transitory computer readable storage mediumhaving program instructions embodied therewith, the program instructionsexecutable by one or more processors to cause the one or more processorsto: receive at least a first plurality of images comprising a firstimage series and a second plurality of images comprising a second imageseries, wherein the first plurality of images and the second pluralityof images together comprise a set of images; analyze the set of imagesto identify values of a first characteristic associated with respectiveimages of the set of images, wherein the first characteristic comprisesa time at which the respective images of the set of images wereobtained; determine at least a second characteristic associated witheach of the images of the set of images; determine an ordering of theimages of the set of images based on: first: values of the firstcharacteristic associated with respective images of the set of images,and second: values of the second characteristic associated withrespective images of the set of images; display a location of a display,a first image of the set of images, that is first in the determinedordering of the images; and in response to input from a user of thecomputing system indicating a direction of movement within the set ofimages, replace, at the location, the first image with a second image,wherein the second image is adjacent to the first image in the orderingof the images.